test(cuda_core): capture machine state on the first CUDA OOM - #2458
Merged
Conversation
Contributor
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
github-actions
Bot
added
the
cuda.core
juenglin
added
P0
Contributor
Author
|
/ok to test |
This comment has been minimized.
This comment has been minimized.
juenglin
marked this pull request as ready for review
July 30, 2026 22:59
kkraus14
approved these changes
Jul 31, 2026
lijinf2
reviewed
Jul 31, 2026
lijinf2
approved these changes
Jul 31, 2026
lijinf2
left a comment
lijinf2
left a comment
Contributor
There was a problem hiding this comment.
This PR leverages pytest_runtest_makereport and pytest_terminal_summary to collect runtime specs of the first error in a sequence of errors of the same type. Good idea!
juenglin
marked this pull request as draft
July 31, 2026 18:28
juenglin
force-pushed
the
oom-diagnostics
branch
from
August 19, 2026 21:25
19115e1 to
3ccde3d
Compare
Contributor
Author
|
/ok to test 3ccde3d |
juenglin
commented
Aug 19, 2026
Contributor
Author
There was a problem hiding this comment.
Example terminal output:
==============================================================================
cuda_core OOM reason checker: first CUDA_ERROR_OUT_OF_MEMORY of this session
==============================================================================
test: tests/test_memory.py::test_device_memory_resource_default_pool
phase: call
pid: 17012
platform: win32
exception: CUDA_ERROR_OUT_OF_MEMORY: The API call failed because it was unable to allocate enough memory or other resources to perform the requested operation.
CUDA_ERROR_OUT_OF_MEMORY means the driver could not obtain some resource;
it is not proof that device memory is exhausted. Creating -- or even just
looking up -- a memory pool first reserves a host virtual-address window
(observed default: about 2x installed device memory) before any device
memory is touched. That reservation can fail while cuMemGetInfo still
reports most of the device free. The probes below check host VA and
physical device memory separately so the two are not confused. Note that
the '2x device memory' figure is an observation from measurement and a
driver source comment, not a documented guarantee -- it can differ across
driver versions and platforms.
--- direct driver probe (bypasses cuda.core's error reporting) ---
cuCtxGetCurrent() -> ok
cuMemGetInfo() -> free=143.50 GiB, total=178.81 GiB (80.3% free)
mempools supported: True
VMM (cuMemAddressReserve) supported: True
small cuMemAlloc(4 KiB) -> ok, freed
allocation granularity: 2097152 bytes
cuMemAddressReserve(2097152 bytes) -> ok, freed
cuMemAddressReserve(384309411840 bytes, observed default pool window, not a documented guarantee) -> <failed: CUresult.CUDA_ERROR_OUT_OF_MEMORY>
cuDeviceGetMemPool(dev 0) -> <failed: CUresult.CUDA_ERROR_OUT_OF_MEMORY>
cuDeviceGetDefaultMemPool(dev 0) -> <failed: CUresult.CUDA_ERROR_OUT_OF_MEMORY>
cuMemPoolCreate(maxSize=2097152) -> ok, destroyed
verdict: likely host VA exhaustion for the pool-sized window only: a capped memory pool (helpers.constants.POOL_SIZE) still creates fine, but a reservation the size of the observed default pool window does not
==============================================================================
(diagnostics also written to /path/to/cuda_core/cuda_core_oom_diagnostics.txt)
juenglin
marked this pull request as ready for review
August 20, 2026 00:00
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue #2381 reported intermittent runs where ~190 tests failed with
CUDA_ERROR_OUT_OF_MEMORY. That issue is now closed: the root cause wastwo driver-managed memory pools (the default device pool and the CUDA graph
pool) that each permanently reserve host virtual-address space sized at
roughly 2x installed device memory, combined with Windows ASLR randomizing
whether two such reservations can find contiguous room. Turning ASLR off on
the affected machine made the suite pass deterministically; see the issue
for the full investigation.
That investigation surfaced a lesson this PR is now built around:
CUDA_ERROR_OUT_OF_MEMORYdoes not mean the device is out of memory. Thedriver returns it whenever it cannot obtain some resource, and creating --
or even just looking up -- a memory pool reserves host virtual address space
before touching any device memory. A pytest log alone cannot distinguish
"host VA exhausted" from "device physical memory exhausted" from "another
process is holding the GPU," and reproducing the original failure required a
specific driver model, so any evidence has to come from whoever hits it next.
This PR keeps that as its purpose: a failure-triggered OOM reason checker
for the next time this error shows up, anywhere, for any reason -- not a
re-diagnosis of #2381, which is already resolved.
What changed since the original version of this PR
nvidia-smi. The original version shelled out tonvidia-smi -qand--query-compute-apps. Per review, this now uses onlycuda.bindings.driverAPIs -- no subprocess, no NVML.call results and leaving the reader to interpret them,
helpers/oom_diagnostics.pynow runs an ordered sequence of probes into a
ProbeSnapshotand turns itinto a one-line verdict via a pure
classify()function:physical device memory exhaustion, host VA exhaustion (at a small or a
pool-sized granularity), default-mempool materialization failure, or
inconclusive.
against the current context/device; there are no OS-specific branches.
Devices without mempools or VMM skip those steps rather than failing.
Alignment comes from
cuMemGetAllocationGranularity(falling back to 2 MiB)rather than a hardcoded page size.
terminalreporter,pytest_terminal_summarypointer) essentiallyunchanged; that part of the design was already reviewed favorably.
They cover the harness and the
classify()decision table with injectedsnapshots (no GPU needed) plus one live smoke test that only exercises the
side-effect-free prefix (context,
cuMemGetInfo, attributes) -- it nevercalls
cuDeviceGetMemPoolor creates a pool, so running the test suitedoesn't itself materialize the ~2x-device-memory reservation this checker
is trying to diagnose.
Checklist